Maj Jason Freels PhD
22 January 2020
“Statistical inference” describes a decision making process wherein statistics and probability theory are used to draw conclusions about some unknown aspect of a population
Decision are made on the basis of a random sample of observations that were drawn from the population
Statistical inference can be sub-divided into two main areas: Estimation and Hypothesis testing
In estimation, a random sample is used to describe some unknown aspect of the population from which the sample was taken
These descriptions come in one of two types - depending on the goal of the analysis
In hypothesis testing, a random sample is used to decide which of two complementary statements about a population are true
A random sample of five of our ‘not-too-exciting’ widgets
Each widget we produce comes with a lot of measureable properties
For each widget, the values associated with each of these properties is based on the materials and processes used to manufacture and assemble the widgets
Moreover, the values measured for each widget will not be the same, but will be uncertain - varying from widget to widget
This uncertainty will be derived from several sources
It’s important that we can make certain assurances - “Our widgets will operate without fault or failure for a specified time period”
- “Our widgets will meet strict dimensional requirements for height, width, weight, etc.” to our customers that the widgets they purchase will meet their requirements.
To make these assurances we need to collect data on our widgets
The data will be collected by conducting different types of tests
Each test we perform on a widget generates one or more observational measures (observations)
Each observation contains information about the properties of the widget being tested
The information obtained from testing each widget gives us a small glimpse of the properties of the overall population of widgets
Pooling the information obtained from all of the widgets together provides a more complete view of the population’s properties
We presume that there exists an underlying mathematical model \(F_{\text{perfect}}\) that perfectly describes the properties of the widgets in the population
This perfect model would also explain the variability that exists in our observed data due the various sources of uncertainty
We’ll never know the true form of \(F_{\text{perfect}}\) as there are also sources of uncertainty we don’t know about and cannot account for
Therefore, our goal is to create an mathematical model \(F_{\text{imperfect}}\) that imperfectly represents \(F_{\text{perfect}}\)
Questions to consider in collecting data
We want to sample from our population of widgets randomly
The \(n\) observations we collect during a test - denoted as \(\xi = [x_1,\ldots,x_n]\) - are realizations of random variables \(\Xi = [X_1,\ldots,X_n]\)
When a random sample contains \(n\) realizations \(x_{1},\ldots,x_{n}\) of \(n\) random variables \(\xi = [x_1,\ldots,x_n]\) we say that the sample has size \(n\) (or that the sample size is \(n\)) and an individual realization \(x_{i}\) is referred to as an observation
Incorrect wording: \(n\) independent realizations of the random variable \(X\)
Correct wording: realizations of \(n\) independent random variables \(X_1,\ldots,X_n\)
In the simplest case the random variables \(X_1,\ldots,X_n\) are independent and have a common distribution function \(F_{_{X}}(x)\) and we say that \(X_1,\ldots,X_n\) are independent and indentically distributed, or IID
IID Example
While in the simplest case \(X_1,\ldots,X_n\) are independent random variables, more complicated cases are possible
In a more general sense we say: A sample \(\xi\) is the realization of a random vector \(\Xi\)
The distribution function of \(\Xi\), denoted by \(F_{_{\Xi}}(\xi)\), is the unknown distribution function that constitutes the object of inference
Therefore, “sample” is just a synonym of “realization of a random vector”. The following examples show how this general definition accommodates the special cases mentioned above
We observe \(n\) realizations \(x_{1},\ldots,x_{n}\) of \(n\) independent random variables \(X_1,\ldots,X_n\) having a common distribution function \(F_{_{X}}(x)\)
The sample is the \(n\)-dimensional vector \(\xi = [x_1,\ldots,x_n]\), which is a realization of the random vector \(\Xi = [X_1,\ldots,X_n]\)
The joint distribution function of \(\Xi\) is
\[ F_{_{\Xi}}(\xi) = F_{_{X}}(x_1)\times\ldots\times F_{_{X}}(x_n) \]
We observe \(n\) realizations \(x_{1},\ldots,x_{n}\) of \(n\) random variables \(X_1,\ldots,X_n\) that are not independent but have a common distribution function \(F_{_{X}}(x)\)
The sample is again the \(n\)-dimensional vector \(\xi = [x_1,\ldots,x_n]\), which is a realization of the random vector \(\Xi = [X_1,\ldots,X_n]\)
However, in this case the joint distribution function \(F_{_{\Xi}}(\xi)\) can no longer be written as the product of the distribution functions of \(X_1,\ldots,X_n\)
Estimation refers to the use statistical procedures to make inferences about a population, based on information obtained from a random sample
There are two basic types of statistical procedures
In the sections below we formally define statistical models and show ways to compute various types of estimates
Throughout the remainder of this presentation we’ll assume that parametric models will be used
We previously defined a random sample \(\xi\) as a realization of a vector of random variables, \(\Xi\)
Analysts use sample statistics to estimate population parameters
The sample mean \(\bar{x}\)The mean computed for a random sample of observed data is used to estimate the unknown mean of the population \(\mu\)
The sample variance \(\operatorname{Var}[x]\)The variance computed for a random sample of observed datais used to estimate population proportions.
An estimate of a population parameter may be expressed in two ways:
Point estimates: population parameter is estimated by a single value of a statistic (i.e. the sample mean ${x} = is a point estimate of the population mean \(\mu\))
Interval estimate: population parameter is estimated by two numbers, between which the parameter is said to lie (i.e. \(a < \hat{\mu} < b\) is an interval estimate of the population mean \(\mu\)
It is desirable that a statistic used as a point estimate for a population parameter be:
Several methods are used to calculate the estimator
A type of interval estimate used to express the uncertainty associated with a particular sampling method
Are preferred to point estimates, because confidence intervals indicate (a) the precision of the estimate and (b) the uncertainty of the estimate
Consists of three parts.
Example: compute an interval estimate of a population parameter
The probability part of a confidence interval is called a confidence level
Describes the likelihood that a particular sampling method will produce a confidence interval that includes the true population parameter.
Interpreting a confidence level
In a confidence interval, the range of values above and below the sample statistic is called the margin of error.
Interpreting the margin of error
Suppose a population of observations exists to which no treatment has been applied
Now suppose that another population exists that’s similar to the control group, except that some treatment has been applied
Samples are drawn from this latter population and the statistics derived from the sample serve as the estimates of the unknown population parameters
A hypothesis statement is created to describe the statistical relationship between the two data sets - this statement contains two elements:
The comparison is deemed statistically significant if the relationship between the data sets would be an unlikely realization of the null hypothesis according to a threshold probability – the significance level
Hypotheses are helpful in determining what outcomes of a study would lead to a rejection of the null hypothesis for a pre-specified level of significance
Distinguishing between the null and alternative hypotheses is aided by considering two types of errors
Hypothesis tests based on statistical significance are akin to expressing confidence intervals
Step 1) Develop the research statement (hypothesis). That is, state what you hope to discover. Interrogative form is common.
Is this hypothesis testable? As attractive as it may be, if a hypothesis cannot be tested, it has little utility to empirical investigators.
Hypotheses are not testable if the concepts (constructs) to which they refer are not adequately (operationally) defined. To facilitate clarity in communication, operational definitions (procedures that define variables) provide the concrete means of evaluating whether a hypothesis constains concepts that can be observed.
Hypotheses are also untestable if they are circular. That is, the event or outcome becomes an explanation for the event. For example: “Your eight-year-old son is distractable in school because he has attention deficit hyperactivity disorder.”
Step 2) Set up the null hypothesis (\(H_0\))
Step 3) Construct the sampling distribution based on the assumption that \(H_0\) is true
Step 4) Compare the sample statistic to the the distribution
Step 5) Reject or fail to reject the \(H_0\) depending on the probability
Several parametric tests exist (e.g., \(z\), \(t\), \(F\), \(\chi^2\))
Discussed that inference is a decision making process
We introduced the types of inference
Walkthrough examples of carrying out various type of inferences